How to update custom environment variables for a Databricks cluster that is not created through the Lazsa Platform

Problem

You can customize a Databricks cluster according to the requirements of your organization by updating the custom environment variables. The environment variables must be updated by the user who has created the Databricks cluster. Now consider that you have a Databricks cluster that is not created through the Lazsa Platform. When you run an API to update the custom variables for the cluster, the API fails. This happens because the API is not run in the context of the user who has created the cluster.

Solution

Since the Databricks cluster is created outside of the Lazsa Platform, it is required for the user who has created the Databricks cluster to complete the following manual steps to update the custom environment variables.

Steps to update the custom environment variables manually

1. Log in to the Databricks instance in the context of the user who has created the Databricks cluster.

2. Click Compute in the left navigation pane. Click the cluster for which you want to update the environment variables.

3. Click the Configuration tab.

   

4. In the Advanced options section under Spark config, update the required Environment variables.